End-to-End Kernel Learning with Supervised Convolutional Kernel Networks

In this paper, we introduce a new image representation based on a multilayer kernelmachine. Unlike traditional kernel methods where data representation is decoupledfrom the prediction task, we learn how to shape the kernel with supervision. Weproceed by first proposing improvements of the recently-introduced convolutionalkernel networks (CKNs) in the context of unsupervised learning; then, we derivebackpropagation rules to take advantage of labeled training data. The resultingmodel is a new type of convolutional neural network, where optimizing the filtersat each layer is equivalent to learning a linear subspace in a reproducing kernelHilbert space (RKHS). We show that our method achieves reasonably competitiveperformance for image classification on some standard “deep learning” datasetssuch as CIFAR-10 and SVHN, and also for image super-resolution, demonstratingthe applicability of our approach to a large variety of image-related tasks.